Hypothesis Testing Part 2
POLS 3316: Statistics for Political Scientists
2023-10-18
Part 1: Hypothesis Testing
Last week:
The *68-95-99.7 Rule: normal distribution probability shorthand
Tying samples and populations
The Central Limit Theorem: get to normal distribution
The Law of Large Numbers: further tie sample to population
Two new, related statistics: standard error and Z-score
The goal was Hypothesis Testing
So what is hypothesis testing?
- We have a theory and we want to see if it’s valid
- We formulate hypotheses (pl.) that would be true if the theory was valid
- We try to disprove or falsify them
- How? By statistically testing data
Review: The Null and Alternative Hypotheses
- The Alternative Hypothesis: This is actually the hypothesis that agrees with our theory. Why?
- The default assumption is that our theory is that there is no effect: The Null Hypothesis
- We can’t prove our hypothesis, but we can get sufficient evidence to reject the null hypothesis
- How? Data and Statistics
How do we reject of confirm the null hypothesis?
How do we reject of confirm the null hypothesis?
- Conduct experiments or make observations to gather data
You know nothing
How do we reject of confirm the null hypothesis?
- Conduct experiments or make observations to gather data
- Compare the results to expectations of random chance
How do we reject of confirm the null hypothesis?
- Conduct experiments or make observations to gather data
- Compare the results to expectations of random chance
- Match random chance: Retain the null hypothesis, reject the alternative hypothesis
How do we reject of confirm the null hypothesis?
- Conduct experiments or make observations to gather data
- Compare the results to expectations of random chance
- Match random chance: Retain the null hypothesis, reject the alternative hypothesis
- Do not match random chance: Reject the null hypothesis - evidence in favor alternative hypothesis
How do we reject of confirm the null hypothesis?
- Conduct experiments or make observations to gather data
- Compare the results to expectations of random chance
- Match random chance: Retain the null hypothesis, reject the alternative hypothesis
- Do not match random chance: Reject the null hypothesis - evidence in favor alternative hypothesis
- How do we do compare to random chance? Tests based on probability and statistics
Test statistics
- Z-score
- t-test
- Chi-square
- ANOVA
- Many others
The Z-score
- Based on the standard normal distribution
- Application of the 69-95-99.7 rule
![]()
68-95-99.7 rule
The Z-score
- Based on the standard normal distribution
- Based on the 69-95-99.7 rule
- Measures how many standard errors a value is from the mean
The Z-score
- Based on the standard normal distribution
- Based on the 69-95-99.7 rule
- Measures how many standard errors a value is from the mean
Fast forward preview: The standard error is a special standard deviation the standard deviation of the sampling distribution of the mean.
The Z-score
Based on the standard normal distribution
Based on the 69-95-99.7 rule
Measures how many standard errors a value is from the mean
Can be used to test:
- hypotheses about the mean of a population
- hypotheses about the difference between two means (two groups with identical distributions)
- hypotheses about the difference between a mean and a value
- hypotheses about the difference between two proportions
Standard error: Tying samples to populations
Standard error: Tying samples to populations
- Quantifies the range around population value (parameter) for the sample value (the statistic)
Standard error: Tying samples to populations
- Quantifies the range around population value (parameter) for the sample value (the statistic)
- Usually the mean but you can figure a standard error for other statistics like the median
Standard error: Tying samples to populations
- Quantifies the range around population value (parameter) for the sample value (the statistic)
- Usually the mean but you can figure a standard error for other statistics like the median
- Distance from the mean is a measure of…
Standard error: Tying samples to populations
- Quantifies the range around population value (parameter) for the sample value (the statistic)
- Usually the mean but you can figure a standard error for other statistics like the median
- Distance from the mean is a measure of…
Dispersion
Standard error: Tying samples to populations
- Quantifies the range around population value (parameter) for the sample value (the statistic)
- Usually the mean but you can figure a standard error for other statistics like the median
- Distance from the mean is a measure of…Dispersion
- If we want to measure disperstion in units equal to the mean, we want to measure dispersion with?
Measure dispersion
- If we want to measure disperstion in units equal to the mean, we want to measure dispersion with?
Standard deviation
The Standard Error of the Mean
- Standard deviation measures dispersion relative to the mean
The Standard Error of the Mean
- Standard deviation measures dispersion relative to the mean
- Standard error measures dispersion between the sample mean and the population mean
The Standard Error of the Mean
- Standard deviation measures dispersion relative to the mean
- Standard error measures dispersion between the sample mean and the population mean
- Standard error of the mean is the sample standard deviation divided by the square root of the sample size:
\(\frac{s}{\sqrt{n}}\)
Stanard Error
Standard Error is the standard deviation of the sample means.
- If we do 1000 trials of random, indendepent, identically distributed variables (random IID variables) from any distribution
- The means of each trial are the sample means
- Central Limit Theorem tells us that the distribution of the sample means will converge to a normal distribution
- If we made a vector of the means of the 10000 trials flipping a coin 20 times and plotted a histogram, it would look like this:
Stanard Error
Standard Error
- Theoretically, if we do an experiment with 500 subjects, it’s one trial with one sample mean. When we start doing hypothesis tests,
- Practically, we can’t do 10000 trials of 500 subjects each. We can only do one trial with 500 subjects, or…
- We use observational data with 500 observations
- We don’t need 500 data points, next week I’ll show you why 30 is sufficient in many cases
Z-Score: Concept
- Number of standard errors from the mean
Z-Score: Concept
- Number of standard errors from the mean
- Probability that actual population parameter is approximately equal to sample statistic
Z-Score: Concept
Number of standard errors from the mean
Probability that actual population parameter is approximately equal to sample statistic
If we know the sample mean, \(\bar{x}\), is 50
standard error, \(\sigma\), is 1
We want to locate the population mean, \(\mu\)
Z-Score: Concept
68-95-99.7 Rule
- 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE
- If SE is 1..
Z-Score: Concept
68-95-99.7 Rule
- 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE
- If SE is 1..
- 99.7% probability that the true population mean is between 47 and 53.
Confidence Interval: Concept
68-95-99.7 Rule
- 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE
Confidence Interval
- The 99.7% Confidence Interval of the sample mean with a sample mean of 50 and standard error of 1 is 47 to 53.
Z-Score: Confidence Interval
Why Z-score instead of 68-95-99 Rule?
- 68-95-99.7 approximates 2 standard deviations for 95%
- Actual value of 2 sd is 95.45%
- Precise Z-value for 95% is 1.96
- Z-score of 2 is still a good mental shortcut for 95% (better than 95%)
- Journal articles publish outcomes with standard errors underneath
Z-scores journal articles
===============================================
Dependent variable:
---------------------------
dist
-----------------------------------------------
speed 3.932***
(0.416)
Constant -17.579**
(6.758)
-----------------------------------------------
Observations 50
R2 0.651
Adjusted R2 0.644
Residual Std. Error 15.380 (df = 48)
F Statistic 89.567*** (df = 1; 48)
===============================================
Note: *p<0.1; **p<0.05; ***p<0.01
Authorship, License, Credits
![Creative Commons License]()
